MICROPHONE ARRAY SYSTEM 
CROSS REFERENCE TO RELATED APPLICATION 

This is a Divisional Application of U.S. Patent Application Serial No. 09/560,355 filed April 
28, 2000. 

BACKGROUND OF THE INVENTION 

1 . Field of the Invention 

The present invention relates to a microphone array system. In particular, the present 
invention relates to a system that performs various kinds of signal processing with respect to sound 
signals received at each microphone to provide various functions. 

2. Description of the Prior Art 

Hereinafter, a sound signal processing technique that utilizes a conventional technique will 
be described. 

In the case where a plurality of sound sources of a desired signal and noise are present in a 
sound field, high quality enhancement of the desired sound, detection of the direction of the desired 
sound and noise suppression are important issues to be addressed for sound signal processing. 
Possible applications that utilize sound signal processing are in a wide range, such as animation and 
sound recording, systems for voice memo, hand-free telephones, teleconference systems, guest- 
reception systems or the like. In order to realize processing for enhancing a desired signal, 
suppressing noise and detecting the direction of the sound source, various sound signal processing 
techniques are under development. 

Conventionally, microphones suitable for a particular application are used to obtain input 
sound signals for use in the processing for enhancing a desired signal, suppressing noise and 
detecting the direction of the sound source. For a compact video camera, a stereo microphone of MS 
(mid-side) system is widely used. In recent years, a unidirectional microphone is used in a personal 
computer that utilizes sound input in application software such as a voice memo, so that a suitable 
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and articulate input sound signal can be obtained. Although these microphone are suitably used in 
view of the use and the cost, they are intended for a single use so that the directivity or the use is 
predetermined. Moreover, the processing of the sound signals received at the microphones is limited 
to the sound signal processing required by the application. 
5 In an apparatus such as a conventional video camera or sound-inputtable personal computer 

that requires microphones suitable to each application and implements only sound signal processing 
required by the application that currently runs, the microphone and the sound signal processing 
function are each intended for a single function. However, for the apparatus designed to have a large 
number of functions, more flexible directionally received sound processing, sound source direction 

10 detecting processing and noise suppressing processing are desirable, and a function that has not 
conventionally required may be required in an application. In this case, since the configuration of 
the apparatus using the conventional microphone with a single function cannot meet this need, it is 
necessary to replace the microphone by a microphone suitable to the required function and also to 
replace the sound signal processing part for received sound signals by another one having the 

1 5 required function. 

As the utilization system is varied, combining a plurality of kinds of sound signal processing 
such as directionally received sound processing, sound source direction detecting processing, noise 
suppressing processing and the like may be needed. In this case, it is necessary to prepare a plurality 
of microphones, each of which has a single function, and to perform sound signal processing for each 

20 individual microphone, and then perform sound signal processing of the combined results from the 
plurality of microphones. Thus, this conventional system requires a large number of microphones, 
so that it results in a large-scale apparatus. Furthermore, it may be difficult to physically arrange the 
required number of microphones to perform a plurality of kinds of sound signal processing in the 
necessary directions. 

25 
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SUMMARY OF THE INVENTION 

Therefore, with the foregoing in mind, it is an object of the present invention to provide a 
microphone array system that eliminates the replacement of the microphones and the replacement 
of the sound signal processing parts, which are conventionally required, regardless of the variation 
5 of the application or the sound signal processing function. It is another object of the present 
invention to achieve a sound signal processing function performing a combination of various kinds 
of sound signal processing in the same microphone arrangement. 

A microphone array system using a unit having a signal processing function such as a 
personal computer as the platform includes at least one microphone arranged along each axis 

10 direction; and a received sound signal processing part for performing signal processing of sound 
signals received at the plurality of microphones, having a directional sound signal calculating 
function for calculating a directional sound signal to an arbitrary direction based on the received 
sound signal with a unidirectivity or bidirectivity pattern along the axis direction, and further having 
at least one function of other sound signal processing functions at the same time. It is preferable that 

1 5 the other sound signal processing functions includes a sound source direction detecting function and 
a noise suppressing function. 

This embodiment achieves a microphone array system including a plurality of microphones 
using a personal computer and allows the system to have a plurality of sound signal processing 
functions including the function for calculating a directional sound signal to an arbitrary direction, 

20 the sound source direction detecting function and the noise suppressing function based on the 
processing of sound signals received at the microphone array. 

In one embodiment, the plurality of microphones are non-directional microphones, at least 
two non-directional microphones are arranged in a first axis direction, and at least two non- 
directional microphones are arranged in a second axis direction that is orthogonal to the first axis. 

25 This makes it possible that the received sound signal processing part has a function for calculating 



a directional sound signal to an arbitrary direction based on a unidirectional estimated sound signal 
to a positive direction on the first axis and a bidirectional estimated sound signal to positive and 
negative directions on the second axis . In another embodiment, the plurality of microphones are 
unidirectional microphones, a first unidirectional microphone is directed to a positive direction on 
5 a first axis, and second and third unidirectional microphones are directed to positive and negative 
directions on a second axis that is orthogonal to the first axis. This makes it possible that the 
received sound signal processing part has a function for calculating a directional sound signal to an 
arbitrary direction based on a unidirectional received sound signal to a positive direction on the first 
axis and a bidirectional received sound signal to positive and negative directions on the second axis. 

10 In still another embodiment, the plurality of microphones are at least one unidirectional microphone 
and at least one bidirectional microphone, the unidirectional microphone is directed to a first axis 
direction, and the bidirectional microphone is directed to a second axis direction that is orthogonal 
to the first axis direction. This makes it possible that the received sound signal processing part has 
a function for calculating a directional sound signal to an arbitrary direction based on a unidirectional 

1 5 received sound signal to a positive direction on the first axis and a bidirectional received sound signal 
to positive and negative directions on the second axis. Furthermore, it is possible that the received 
sound signal processing part has a sound source direction detecting function for detecting a sound 
source direction, using a power in each axis direction of a sound signal calculated by the directional 
sound signal calculating function and cross-correlation thereof. 

20 The microphone array system of the present invention can have the function for calculating 

a directional sound signal to an arbitrary direction and further have sound signal processing functions 
such as the function for detecting a sound source direction and the function for suppressing noise 
based on a plurality of kinds of processing of sound signals received at the microphone array by 
providing a plurality of microphones on a personal computer, which is the platform, regardless of 

25 the application or the sound signal processing function. 



The microphone array system of the present invention can have the function for calculating 
a directional sound signal to an arbitrary direction based on a unidirectional estimated sound signal 
to the positive direction of the first axis and a bidirectional estimated sound signal to the positive and 
negative directions of the second axis. 
5 The microphone array system of the present invention can have the sound source direction 

detecting function for detecting the sound source direction using the powers of the sound signals on 
the axes that are calculated by the directional sound signal calculating function and the cross- 
correlation coefficient therebetween. 

These and other advantages of the present invention will become apparent to those skilled 
10 in the art upon reading and understanding the following detailed description with reference to the 
accompanying figures. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG 1 is a diagram showing an example of the configuration of a microphone array where 
a plurality of microphones are arranged along the axis direction using a personal computer as the 
1 5 platform of the present invention. 

FIG 2 is a diagram showing an example of a configuration different from that of FIG. 1 of 
a microphone array of the present invention. 

FIG 3 is a diagram illustrating the principle of processing for calculating directional sound 
signals by the microphone array system of the present invention. 
20 FIG 4 is a diagram showing an example of the configuration of a directional sound signal 

calculating part 50. 

FIGs. 5 A and 5B are diagrams showing a received sound signal with a unidirectivity pattern 
to the negative direction on the X axis and a received sound signal with a bidirectivity pattern to the 
positive and negative directions on the Y axis obtained by the microphone array system of the present 
25 invention. 
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FIGs. 6A and 6B are diagrams showing a received sound signal with a directivity pattern for 
left channel signal reception and a received sound signal with a directivity pattern for right channel 
signal reception of two channel stereo sound reception that are estimated by the microphone array 
system of the present invention, respectively 
5 FIG 7 shows an example of the configuration of the sound source direction detecting part 

60. 

FIGs. 8A to 8E are diagrams showing a received sound signal with a unidirectivity pattern 
processed by the subtracter 1 2 1 and a received sound signal with a bidirectivity pattern processed by 
the subtracter 122 with respect to an impulse sound source from the negative direction on the X axis 
10 according to the microphone array system of the present invention. 

FIGs. 9A to 9E show a received sound signal with a unidirectivity pattern and a received 
sound signal with a bidirectivity pattern with respect to an impulse sound sources from the direction 
of 90° with respect to the negative direction on the X axis according to the microphone array system 
of the present invention. 

1 5 FIGs. 1 OA to 1 0E show a received sound signal with a unidirectivity pattern and a received 

sound signal with a bidirectivity pattern, with respect to an impulse sound sources from the direction 
of 1 80° with respect to the negative direction on the X axis according to the microphone array system 
of the present invention. 

FIGs. 1 1 A to 1 IE show a received sound signal with a unidirectivity pattern and a received 

20 sound signal with a bidirectivity pattern, with respect to an impulse sound sources from the direction 
of 270° with respect to the negative direction on the X axis according to the microphone array system 
of the present invention. 

FIG 12 is a diagram showing the pattern classification of sound source directions by the 
comparison of the power ratio P of the unidirectivity and the bidirectivity and the threshold Tp and 
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the comparison of the cross-correlation coefficient and the thresholds TR1 and TR2 according to the 
microphone array system of the present invention. 

FIG 13 is a diagram showing an example of the configuration of the microphone array 
system of Embodiment 2 of the present invention. 
5 FIG 14 is a schematic diagram showing an example of the basic configuration of the 

microphone array system of Embodiment 3 of the present invention. 

FIG 15 is a schematic diagram showing an example of the basic configuration of the 
microphone array system of Embodiment 4 of the present invention. 

FIG 16 is a diagram showing the adjustment of the delay sampling number of the delay 
10 units and the gain amount of the gain circuits based on the camera image capturing direction of 
Embodiment 4 of the present invention. 

FIG 1 7 shows an example of the configuration of the directional sound signal calculating part 
50c of Embodiment 4 of the present invention. 

FIG 1 8 shows an example of the configuration of the sound source direction detecting part 
1 5 60c of Embodiment 4 of the present invention. 

FIG 19 is a schematic diagram showing an example of the basic configuration of the 
microphone array system of Embodiment 5 of the present invention. 
DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Hereinafter, the embodiments of the microphone array system of the present invention will 
20 be described with reference to the accompanying drawings. 
Embodiment 1 

The microphone array system of Embodiment 1 includes a microphone array where a plurality 
of microphones are arranged along the axis direction, using a personal computer as the platform. The 
system performs signal processing of sound signals received at these microphones to generate 
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received sound signals with a unidirectivity or bidirectivity pattern along the axis direction. The 
system includes a directional sound signal calculating function for calculating a directional sound 
signal to an arbitrary direction based on the generated received sound signals, and further include a 
sound source direction detecting function, a noise suppressing function and a sound signal processing 
function. 

FIG 1 is a diagram showing an example of the configuration of a microphone array where 
a plurality of microphones are arranged along the axis directions, using a personal computer as the 
platform. In this example, two orthogonal axes of the X axis and the Y axis as shown in FIG. 1 are 
used as the axis. Three axes of X, Y and Z can be used, and the axes are not necessarily orthogonal. 

A microphone array section 1 0 includes a plurality of microphones 1 1 arranged on the X axis 
and a plurality of microphones 12 arranged on the Y axis. The microphones 1 1 and 12 can be either 
non-directional, unidirectivity or bidirectional microphones. A sound signal received from each 
microphone is sent through analog microphone interfaces including a connector 20, a microphone 
amplifier 21 , a two channel analog-digital converter 30 (hereinafter, referred to as "AD converter"), 
and thus the received sound signals are connected to a directional sound signal calculating part 50, 
a sound source direction detecting part 60, and a noise suppressing part 70 via a bus 40 of the 
platform personal computer. The directional sound signal calculating part 50, the sound source 
direction detecting part 60, and the noise suppressing part 70 can be an independent device dedicated 
to the particular function, or can be designed as a processing program that is described so that the 
particular function is realized by the central processing unit (hereinafter, referred to as CPU) and the 
memory of the platform personal computer. 

FIG 2 is a diagram showing a microphone array with a configuration different from that of 
FIG. 1. In this example, a USB (universal serial bus) interface is used as the interface of the 
microphone. In this example as well, two axes of X and Y as shown in FIG 1 are used as the axis. 
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In the example shown in FIG. 2, the microphones 1 1 and 12 of the microphone array section 10 can 
be arranged in the same manner as in the example in FIG 1 . Each of the microphones 1 1 and 1 2 is 
connected to a bus 40 via a USB hub 90, a connector 20a and a USB interface 91 and connected to 
a directional sound signal calculating part 50, a sound source direction detecting part 60, and a noise 
5 suppressing part 70. 

All of these functions are not necessarily provided in the system. For example, the directional 
sound signal calculating part and only one other function may be combined. Alternatively, all the 
functions may be combined, and other sound signal processing functions can be added thereto. 

Next, sound signal processing of the directional sound signal calculating function, the sound 
10 source direction detecting function, the noise suppressing function of the microphone array system 
of the present invention will be described with reference to the arrangement examples of the 
microphones. 

In a microphone array section 10a of the example shown in FIG. 3, four non-directional 
microphones 100a to lOOd are arranged along the positive and negative directions of the X and Y 

1 5 axes, each microphone corresponding to one direction, to receive sound signals. The direction of the 
front of the microphone array system corresponds to the negative direction on the X axis. The 
microphones 1 00a to 1 OOd are positioned close to each other. In this example, the distance between 
the microphones 1 00a and 1 00c and the distance between the microphones 1 00b and 1 OOd are a value 
obtained by dividing the sound velocity by the sampling frequency. A delay unit 110 performs 

20 processing of delaying for one sampling period and is connected to the microphone 1 00c. Numerals 
121 and 122 denote subtracters. 

The directional sound signal calculating function will be described primarily from the aspect 
of the directional sound signal calculating part 50. FIG 4 is a diagram showing an example of the 
configuration of a directional sound signal calculating part 50. 
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In the first stage, the directional sound signal calculating function generates a received sound 
signal from a microphone whose directivity has a unidirectivity pattern to the negative direction on 
the X axis and a received sound signal from a microphone whose directivity has a bidirectivity 
pattern to the positive and negative directions on the Y axis. Next, in the second stage, the 
directional sound signal calculating function estimates a left (L) channel signal and a right (R) 
channel signal having the directivity to a particular direction, based on the received sound signals 
with a unidirectivity pattern to the negative direction on the X axis and the received sound signal 
with a bidirectivity pattern to the positive and negative directions on the Y axis. 

First, the process in the first stage will be described. 

As shown in FIG. 3, the subtracter 1 2 1 subtracts the received sound signal of the microphone 
100c that is delayed by one sampling period by the delay unit 110 from the received sound signal of 
the microphone 100a. As a result, a received sound signal having a unidirectivity pattern to the 
negative direction on the X axis as shown in FIG. 5 A is generated. Furthermore, the subtracter 122 
subtracts the received sound signal of the microphone lOOd from the received sound signal of the 
microphone 100b. As a result, a received sound signal having a bidirectivity pattern to the negative 
and positive directions on the Y axis as shown in FIG. 5B is generated. In FIG. 5B, the positive 
direction on the Y axis corresponds to the plus directivity, and the negative direction on the Y axis 
corresponds to the minus directivity. 

Next, the process in the second stage will be described. 

The process for generating a received sound signal having a directivity pattern to the left 
channel direction will be described below. As shown in FIG. 4, the received sound signal having a 
unidirectivity pattern of FIG 5 A, which is an output signal from the subtracter 121, and the received 
sound signal having a bidirectivity pattern of FIG. 5B, which is an output signal from the subtracter 
122, are input to a subtracter 123 of the directional sound signal calculating part 50, and the latter 



10 



is subtracted from the former. This subtraction provides a received sound signal having a directivity 
pattern for left channel signal reception for two channel stereo sound reception as shown in FIG. 6A. 
FIG 6 A shows a directivity pattern having an angle of about 45° with respect to the front direction. 
However, this angle is adjustable so that a directivity pattern to an arbitrary direction can be obtained. 
In other words, a directivity pattern to an arbitrary direction can be obtained by adjusting the gains 
of the output signals from the subtracters 1 2 1 and 1 22, and then inputting the results to the subtracter 
123 for a subtraction process. For example, the gain of the output signal from the subtracter 121 is 
enlarged, and the gain of the output signal from the subtracter 122 is reduced, and then the subtracter 
123 subtracts the latter from the former. In this case, the directivity of the obtained directivity 
pattern becomes near the front direction, compared to the directivity pattern shown in FIG 6A. 

The process for generating a received sound signal having the directivity pattern to the right 
channel direction will be described below. A received sound signal having a unidirectivity pattern 
of FIG 5 A, which is an output signal from the subtracter 121, and a received sound signal having 
a bidirectivity pattern of FIG 5B, which is an output signal from the subtracter 122, are input to an 
adder 124, and the former and the latter are added. This addition provides a received sound signal 
having a directivity pattern for right channel signal reception for two channel stereo sound reception 
as shown in FIG 6B. Similarly to the case of the left channel, it is possible to adjust the angle of the 
pattern with the plus directivity and the minus directivity. 

Next, the sound source direction detecting function will be described primarily from the 
aspect of the sound source direction detecting part 60. The sound source direction detection is 
performed by utilizing the powers of the received sound signal with the unidirectivity pattern to the 
negative direction on the X axis (front direction) and the received sound signal with the bidirectivity 
pattern to the positive and negative directions on the Y axis, and the cross-correlation coefficient 
therebetween. 
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FIG 7 shows an example of the configuration of the sound source direction detecting part 
60. The sound source direction detecting part 60 includes a power ratio calculating part 1 30, a cross- 
correlation coefficient calculating part 140, and a determining part 61 . The sound source direction 
detecting part 60 receives a received sound signal having a unidirectivity pattern to the negative 
direction on the X axis as shown in FIG. 5 A from the subtracter 121 and a received sound signal 
having a bidirectivity pattern to the positive and negative directions on the Y axis as shown in FIG 
5B from the subtracter 122. 

For simplification for description of the basic principle of the sound source direction 
detection, it is assumed that the sound input signal is an impulse signal. FIG. 8 A shows a received 
sound signal with a unidirectivity pattern processed by the subtracter 121, and FIG. 8B shows a 
received sound signal with a bidirectivity pattern processed by the subtracter 122 with respect to an 
impulse sound source from the direction of 0° (front direction) in the negative direction on the X 
axis. In the same manner, FIGs. 9A, 1 OA, and 1 1 A show received sound signals with a unidirectivity 
pattern processed by the subtracter 121, and FIGs. 9B, 10B, and 1 IB show a received sound signals 
with a bidirectivity pattern processed by the subtracter 122, with respect to an impulse sound sources 
from the directions of 90°, 180°, and 270°, respectively, in the negative direction on the X axis. 

The power ratio calculating part 130 calculates the ratios of the powers of the output signals 
from the subtracters 121 and 122, namely, the powers with respect to each of the received sound 
signals of FIGs. 8 A and 8B to 1 1 A and 1 IB. In these figures, the diagram C shows the power of the 
received sound signal with a unidirectivity pattern processed by the subtracter 121, and the diagram 
D shows the power of the received sound signal with a bidirectivity pattern processed by the 
subtracter 122. 

Next, the cross-correlation coefficient calculating part 140 calculates the cross-correlation 
coefficient between the received sound signal with a unidirectivity pattern processed by the subtracter 
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121 and the received sound signal with a bidirectivity pattern processed by the subtracter 122 
FIGs. 8 A and 8B to 1 1 A and 1 IB. The cross-correlation coefficient R can be calculated with the 
following equation. 



Equation 1 

t 



Equation 1 
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where m(ti) is a signal from the subtracter 121 and n(ti) is a signal from the subtracter 
122, and 1 is the sampling number for calculation of the cross-correlation coefficient, and 
generally is a value more than several hundreds . 

The cross-correlation coefficient R calculated in Equation 1 is from 
-1.0 to 1.0, and shows how similar the two signals m(ti) and n(ti) are. For example, the cross- 
correlation coefficient shows the followings. 

In the case of R = 1.0, m(ti) and n(ti) have the same amplitude and the phase (the signals 
having the same waveforms). 

In the case of R = 0.0, m(ti) and n(ti) are not correlated (not similar at all). 

In the case of R = - 1 .0, m(ti) and n(ti) have the same amplitude and the opposite phase (the 
sign of the amplitude of the signals is opposite). 

In FIGs. 8 to 11, the diagram E shows the result of calculating the cross-correlation 
coefficient according to Equation 1 . 

Now, the sound source direction is estimated by using the ratio of the power of the received 
sound signal with a unidirectivity pattern and the power of the received sound signal with a 
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bidirectivity pattern and the cross-correlation coefficient therebetween. For example, the sound 
source direction can be estimated by determining which direction of 0°, 90°, 1 80° or 270° the sound 
source outputting the impulse is in, where the 0° direction corresponds to the negative direction on 
the X axis. This processing method will be described below. 

First, the power ratio P of the unidirectivity and the bidirectivity is obtained. More 
specifically, P = (the power of the received sound signal with a bidirectivity pattern) / (the power of 
the received sound signal with a unidirectivity pattern) is obtained. Next, thresholds Tp, TR1 and 
TR2 as shown below are introduced so that the power ratio P of the unidirectivity and the 
bidirectivity and Tp are compared, and the cross-correlation coefficient R and TR1 and TR2 are 
compared. Herein, Tp is a positive value, TR1 is a negative value and TR2 is a positive value, and 
four patterns as shown in FIG 12 are obtained by setting suitable thresholds, as described later. 

In the examples with respect to the impulse sound source shown in FIGs. 8 to 11, if the 
thresholds are Tp= 0. 1 , TR1 = - 0.2, and TR2 = 0.2, it can be estimated which direction of the sound 
source direction of 0°, 90°, 180° or 270° the sound source is in 

Furthermore, in the processing for estimating the sound source direction, the sound source 
direction can be obtained by a method other than the above-described method of determination with 
the thresholds. For example, if values corresponding to various directions from 0 to 360° of the 
sound source are previously obtained by using the power ratio P of the sound signal with a 
bidirectivity to the sound signal with a unidirectivity and the cross-correlation coefficient R as the 
parameters, the sound source direction can be determined based on the two parameters of the actually 
measured power ratio P of the bidirectivity to the unidirectivity and cross-correlation coefficient R. 

Next, the noise suppressing function in a noise suppressing part 70 will be described. Noise 
can be erased by mutual subtraction of received sound signal components in the noise source 
direction among the received sound signals from the microphones. The sound source direction 



14 



detecting part 60 can estimate the desired sound source direction, so that it is certainly possible that 
the noise component in the directions other than the desired sound source direction can be suppressed 
by directing the directivity to the direction of the desired sound source. 

As described above, the microphone array system including a plurality of microphones on 
a personal computer, which is the platform, of the present invention can utilize selectively the 
functions of the directional sound signal calculating part 50, the sound source direction detecting part 
60, and the noise suppressing part 70. Moreover, a plurality of functions can be utilized at the same 
time. 

Embodiment 2 

Similarly to the microphone array system of Embodiment 1 , the microphone array system of 
Embodiment 2 includes a microphone array where a plurality of microphones are arranged along the 
axis directions, using a personal computer as the platform. The system performs signal processing 
of sound signals received at these microphones to generate received sound signals with a 
unidirectivity or bidirectivity pattern along the axis direction. The system includes a directional 
sound signal calculating function for calculating a directional sound signal with respect to an 
arbitrary direction based on the obtained received sound signals, and further include a plurality of 
sound signal processing functions including a sound source direction detecting function and a noise 
suppressing function. However, the microphone array system of Embodiment 1 is different from that 
of Embodiment 2 in that the non-directional microphones in Embodiment 1 are replaced by a 
plurality of unidirectional microphones in Embodiment 2. 

FIG 13 is a diagram showing an example of the configuration of the microphone array 
system of Embodiment 2. A microphone array section 1 Ob includes three unidirectional microphones 
200a to 200c arranged in the negative direction on the X axis, and the positive and negative 
directions on the Y axis, namely, in the directions of 0°, 90° and 270°, respectively, so as to obtain 
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received sound signals. The front direction of the microphone array system is set to be the negative 
direction on the X axis. In Embodiment 2, although a received sound signal with the unidirectivity 
pattern with respect to the direction of 0° is obtained, it is necessary to generate received sound 
signals with a bidirectivity pattern with respect to the positive and negative directions on the Y axis. 
The directional sound signal calculating part 50a, the sound source direction detecting part 60a, and 
the noise suppressing part 70a of Embodiment 2 have the following configurations. Numeral 122a 
denotes a subtracter. 

In the first stage in the processing for calculating directional sound signals, a sound signal 
received from a microphone having a bidirectivity pattern to the positive and negative directions on 
the Y axis is generated. Next, in the second stage, a left (L) channel signal and a right (R) channel 
signal having a directivity to a specific direction are calculated based on the received sound signal 
with the unidirectivity pattern to the negative direction on the X axis and the received sound signal 
with the bidirectivity pattern to the positive and negative directions on the Y axis. 

The process in the first stage will be described. The sound signal received from a microphone 
having a bidirectional pattern to the positive and negative directions on the Y axis is generated in the 
following manner. The subtracter 122a subtracts the received sound signal of the microphone 200c 
from the received sound signal of the microphone 200b. As a result, the received sound signal 
having a bidirectivity pattern to the negative and positive directions on the Y axis as shown in FIG. 
5B is generated. 

The process for calculating the left (L) channel signal and the right (R) channel signal in the 
second stage is the same as that in Embodiment 1, except that the input signal from the subtracter 
121 in FIG. 4 of Embodiment 1 is replaced by an input signal from the unidirectional microphone 
200a, and the input signal from the subtracter 1 22 in FIG. 4 of Embodiment 1 is replaced by an input 
signal from the subtracter 122a. Similarly to Embodiment 1, the result of subtracting the received 
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sound signal with the bidirectivity pattern from the received sound signal with the unidirectivity 
pattern by the subtracter 123 is used as the left channel signal. The result of adding the received 
sound signal with the unidirectivity pattern and the received sound signal with the bidirectivity 
pattern by the adder 124 is used as the right channel signal. 

The process of the sound source direction detecting part 60a and the process of the noise 
suppressing part 70a are the same as those in Embodiment 1, and therefore is omitted, where 
appropriate. 

As shown in FIG. 13, each of the functions of the directional sound signal calculating part 
50a, the sound source direction detecting part 60a, and the noise suppressing part 70a can be utilized 
together with the directional sound signal calculating function or other functions at the same time. 
Embodiment 3 

The microphone array system of Embodiment 3 includes a microphone array where a plurality 
of microphones are arranged along the axis directions, using a personal computer as the platform. 
The system performs signal processing of sound signals received at these microphones to generate 
received sound signals with a bidirectivity pattern along the axis direction. The system includes a 
directional sound signal calculating function for calculating a directional sound signal with respect 
to an arbitrary direction based on the obtained received sound signals, and further include a sound 
signal processing function such as a sound source direction detecting function and a noise 
suppressing function. In Embodiment 3, unidirectional microphones and bidirectional microphones 
are used. 

FIG 14 is a diagram showing an example of the configuration of the microphone array 
system of Embodiment 3. A microphone array section 10c includes a unidirectional microphone 
200d having a directivity to the negative direction on the X axis (direction of 0°) and a bidirectional 
microphone 300a having directivities to the positive and negative directions on the Y axis (direction 
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of 90° and 270°), so as to obtain received sound signals. In Embodiment 3, a received sound signal 
with the unidirecti vity pattern with respect to the direction of 0° and a received sound signal with the 
bidirectivity pattern to the positive and negative directions on the Y axis are obtained from the 
microphones 200d and 300a. Therefore, there is no need of providing subtracters corresponding to 
the subtracters 121 and 122 in Embodiment 1 and the subtracter 222 in Embodiment 2. The 
directional sound signal calculating part 50b, the sound source direction detecting part 60b, and the 
noise suppressing part 70b are provided. 

The process for calculating the left (L) channel signal and the right (R) channel signal by the 
directional sound signal calculating part 50b is the same as those in Embodiments 1 and 2, and also 
is the same as that of a conventional MS microphone, except the input signals as follows. The input 
signal from the subtracter 121 in FIG. 4 of Embodiment 1 is replaced by an input signal from the 
unidirectional microphone 200d, and the input signal from the subtracter 122 in FIG. 4 of 
Embodiment 1 is replaced by an input signal from the bidirectional microphone 300a. Similarly to 
Embodiment 1 , the result of subtracting the received sound signal with the bidirectivity pattern from 
the received sound signal with the unidirecti vity pattern by the subtracter 123 is used as the left 
channel signal. The result of adding the received sound signal with the unidirecti vity pattern and the 
received sound signal with the bidirectivity pattern by the adder 124 is used as the right channel 
signal. 

The process of the sound source direction detecting part 60b and the process of the noise 
suppressing part 70b are the same as those in Embodiment 1, and therefore is omitted, where 
appropriate. 

Also in Embodiment 3, as shown in FIG. 14, each of the functions of the directional sound 
signal calculating part 50b, the sound source direction detecting part 60b, and the noise suppressing 
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part 70b can be utilized together with the directional sound signal calculating function or other 
functions at the same time. 
Embodiment 4 

The microphone array system of Embodiment 4 includes a camera and a microphone array 
where a plurality of microphones are arranged along the axis directions, using a personal computer 
that controls the movable camera as the platform. The system performs signal processing of sound 
signals received at these microphones to generate received sound signals with a unidirectivity pattern 
or bidirectivity pattern along the axis directions. The system includes a directional sound signal 
calculating function for calculating a directional sound signal with respect to an arbitrary direction 
based on the obtained received sound signals. This embodiment provides a simple method for 
adjusting the directivity pattern of the microphones, which is performed by adjusting the delay 
sampling number and the gain of a delay unit. 

FIG 15 is a diagram showing an example of the configuration of the microphone array 
system of Embodiment 4. 

A microphone array section 1 0a includes non-directional microphones 1 00a to 1 OOd having 
directivities to the negative direction on the X axis (0°), the positive direction on the Y axis (90°), 
the positive direction on the X axis (180°) and the negative direction on the Y axis (270°). The 
outputs of the microphones 100a to lOOd are connected to delay units 110a to HOd, respectively. 
The outputs of the delay units 1 10a to 1 lOd are connected to gain units 1 50a to 1 50d, respectively. 
A movable camera 160 is rotated at any angle from 0° to 360° so that the directions in which the 
camera takes an image (hereinafter, referred to as "camera image capturing direction") can be 
changed. For convenience, the camera can be rotated at an angle of either one of 0°, 90°, 1 80° and 
270°. A camera-orientation detector 170 detects the image capturing direction of the camera 160. 
For example, the orientation of the camera can be detected by presetting the reference direction of 
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the axis of the housing of the camera with respect to the camera stand and detecting the amount of 
the rotation from the preset direction. A delay sampling number adjusting part 180 adjusts so that 
the delay sampling number of each of the delay units 1 1 0a to 1 1 Od corresponds to the delay sampling 
number shown in FIG. 16 based on the camera image capturing direction detected by the camera- 
5 orientation detector 1 70. A gain amount adjusting part 1 90 adjusts so that the amount of the gain of 
each of the gain units 150a to 150d corresponds to the amount of the gain shown in FIG. 16 based 
on the camera image capturing direction detected by the camera-orientation detector 170. 
Furthermore, as described later, the gain amount adjusting part 190 adjusts the gain amounts of gain 
units 150e and 150f in the directional sound signal calculating part 50c. 

1 0 An adder 121c adds the output signal from the microphone 1 00a and the output signal from 

the microphone 100c that have been subjected to the delay and gain processes, and an adder 122c 
adds the output signal from the microphone 100b and the output signal from the microphone lOOd 
that have been subjected to the delay and gain adjustment. 

Next, FIG. 17 shows an example of the configuration of the directional sound signal 

15 calculating part 50c. The directional sound signal calculating part 50c includes gain units 150e to 
15 Oh so that adjustment of the gain amount of +1.0 or -1.0 is performed in accordance with the 
image capturing direction of the camera 160, unlike the directional sound signal calculating part 50 
in FIG. 4. The gain units 1 50e to 1 50h are adjusted by the gain amount adjusting part 1 90 so that the 
gain amounts thereof corresponds to those shown in FIG 16. Numerals 123c and 124c are adders, 

20 and are the same as the adder 124 in FIG 4. 

The output from the adder 1 23c is used as the left channel output signal, and the output from 
the adder 124c is used as the right channel output signal. 

The delay sampling number of the delay units and the gain amount of the gain units with 
respect to the orientation of the camera provide the following advantages. Regarding the adjustment 
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of the delay units, the delay sampling number of the delay unit connected to the non-directional 
microphone arranged farthest from the orientation of the camera (that is, the delay unit 150c in the 
case where the camera image capturing direction is 0°, and the delay unit 1 50d in the case where the 
camera image capturing direction is 90°) is set to be 1, and the delay sampling number of the other 
5 delay units is set to be 0. Therefore, regardless of the orientation of the camera, either 0°, 90°, 1 80° 
or 270°, this configuration is equivalent to that of Embodiment 1 in FIG. 3 from the aspects of the 
sound source direction and the arrangement of the non-directional microphones and the delays. Next, 
regarding the gain adjustment of the gain units 1 50a to 1 50d, the gain amount of the gain units 1 50a 
to 150d is +1.0 or -1.0, which is determined so that the functions of the adder 121c and 122c are 
10 equivalent to the subtraction process by the adders 121 and 122inFIG. 3, regardless of the direction 
of the camera. 

Furthermore, regarding the gain units 150e to 150h in the directional sound signal 
calculating part 50c, the gain amounts are adjusted so that the operations of the adders 123c and 1 24c 
are equivalent to the subtraction process by the subtracter 123 and the addition process by the adder 
15 124 of Embodiment 1 in FIG 4, regardless of the orientation of the camera, respectively. 

Thus, regardless of the image capturing direction of the movable camera, either 0°, 90°, 
180° or 270°, the directional sound signal calculating part 50c that functions in the same manner as 
the directional sound signal calculating part 50 of Embodiment 1 can be obtained by adjusting the 
delay sampling number of the delay units 110a to 11 Od and the gain amount of the gain units 150a 
20 to 150h. 

Next, the configuration of the sound source direction detecting part 60c will be described. 
The sound source is detected in the same manner as in Embodiment 1 , which utilizes the cross- 
correlation coefficient of the powers of the received sound signal with a unidirectivity pattern to the 
front direction of the camera and the received sound signal with a bidirectivity pattern to the positive 
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and negative directions on the Y axis. However, in this embodiment, the delay sampling number and 
the gain amount of the delay units are adjusted. 

FIG 1 8 shows an example of the configuration of the sound source direction detecting part 

60c. 

5 The sound source direction detecting part 60c includes a power ratio calculating part 1 30c 3 

a cross-correlation coefficient calculating part 140c, and a determining part 61c. As shown in FIG. 
18 ? the output signals from the adders 121c and 122c are input to the power ratio calculating part 
130c, and the output signals from the adders 121c and 122c are input to the cross-correlation 
coefficient calculating part 140c. The functions of the components of the sound source direction 

1 0 detecting part 60c have the functions of the corresponding components of the sound source direction 
detecting part 60 of Embodiment 1, and therefore will not be described further. 

Thus, regardless of the image capturing direction of the movable camera, either 0°, 90°, 
180° or 270°, the sound source direction detecting part 60c allows detection of whether or not the 
sound source is in the direction of the orientation of the camera. 

15 The noise suppressing part 70c can have the same configuration as that of Embodiment 1 

where the direction of the orientation of the camera is set to be the camera front by adjusting the 
delay sampling number and the gain amount in accordance with the orientation of the camera 160 
in the same manner. The description thereof is omitted in this embodiment. 
Embodiment 5 

20 The microphone array system of Embodiment 5 includes a camera and a microphone array 

where a plurality of microphones are arranged along the axis directions, using a personal computer 
that controls a video camera as the platform. The system performs signal processing of sound signals 
at received these microphones and has a directional sound signal calculating function for calculating 
a directional sound signal with respect to the camera front direction and a memorandum recording 
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function by the speech of the camera operator (so-called voice memo function) based on the obtained 
received sound signals. 

In Embodiment 5, the sound source is located in either the camera front direction (0° 
direction) of a subject to be shot or the direction of the camera operator (e.g., 1 80° direction). The 
5 direction of the unidirectivity pattern for the directional sound signal calculating function of 
Embodiment 4 is usually set to be 0°, and the direction to be detected by the sound source direction 
detecting function is set to be 1 80°, which is the direction of the camera operator. When the speech 
of the camera operator is detected, namely when the sound source is in the 1 80° direction, the voice 
memo function is turned on so that the spoken sound of the camera operator is recorded. The 

10 directional received sound calculating function, the sound source direction detecting function and 
the sound enhancement processing function with respect to not only 0° and 1 80° as above, but also 
other arbitrary directions can be provided by combining the configurations of Embodiment 4. 

Recording by the voice memo function is performed simply by recording the received sound 
signal with a unidirectivity pattern to the 180° direction. However, this can be performed by 

15 recording received sound signals from a non-directional microphone. In the following example, 
when the speech of the camera operator is detected, the voice memo function is turned on and the 
received sound signal with the unidirectivity pattern to the 180° direction is recorded so that the 
spoken sound of the camera operator is recorded in order to enhance the sound coming from the 180° 
direction. 

20 FIG 19 is a diagram showing an example of the configuration of the microphone array 

system of Embodiment 5. 

Non-directional microphones 100a to lOOd in a microphone array lOd are the same as those 
in Embodiment 4, except that the outputs from microphones 100a and lOOd are processed by two 
systems. Numerals 1 1 Oe and 1 1 Of denote delay units. The delay unit 1 1 Oe delays the received sound 
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signal of a microphone 1 00c by the delay sampling number. The delay unit 1 1 Of delays the received 
sound signal of a microphone 100a by the delay sampling number. Thus, the received sound signal 
processings of the microphones 100a and 100c are performed by two systems in parallel so as to 
generate received sound signals with two patterns of the unidirectivity pattern to the 0° direction and 
5 the unidirectivity pattern to the 180° direction. Subtracters 12 Id and 122d are the same as the 
subtracters 121 and 122 of Embodiment 1, and the results are input to a directional sound signal 
calculating part 50d. On the other hand, a subtracter 121e subtracts the received sound signal of the 
microphone 1 00a that is delayed by one sampling from the received sound signal of the microphone 
1 00c so as to generate a received sound signal with a unidirectivity pattern to the 1 80° direction, and 

1 0 the result is input to a sound source direction detecting part 60d. 

The directional sound signal calculating part 50d is the same as that in FIG 4 of Embodiment 
1, except that the input signal from the subtracter 121 in FIG 4 of Embodiment 1 is replaced by an 
input signal from the subtracter 12 Id, and the input signal from the subtracter 122 in FIG 4 of 
Embodiment 1 is replaced by an input signal from the subtracter 122d. Similarly to Embodiment 1, 

15 the result of subtracting the received sound signal with the bidirectivity pattern from the received 
sound signal with the unidirectivity pattern by the subtracter 123 is used as the left channel signal. 
The result of adding the received sound signal with the unidirectivity pattern and the received sound 
signal with the bidirectivity pattern by the adder 124 is used as the right channel signal. 

The sound source detecting part 60d is the same as that in FIG 7 of Embodiment 1, except 

20 that the input signal from the subtracter 121 in FIG. 7 is replaced by a signal from the subtracter 
121e, and the input signal from the subtracter 122 in FIG. 7 is replaced by a signal from the 
subtracter 122d. 

The sound source detecting part 60d detects whether or not the spoken sound is in the 
direction of the camera operator, namely, whether or not the sound source is in the 180° direction. 
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In the case where the sound source is detected in that direction, a voice memo switch 400 is turned 
on, the signal from the subtracter 1 2 1 d is delivered to a recording part for recording. The signal from 
the subtracter 12 Id has a directivity pattern to the camera operator, and therefore is recorded as a 
speech memorandum. 

As described above, the voice memo of the camera operator can be obtained together with 
good image and recording of the subject of the camera by detecting the sound source by the sound 
source detecting function in the direction of the camera operator (1 80°) while receiving sounds with 
a unidirectivity pattern using the directional sound signal calculating function in the front direction 
of the movable camera (0°). 

In the embodiments of the present invention, the number, the arrangement and the distance 
of microphones of the microphone array system are only illustrative for convenience and not limited 
to particular values. 

The invention may be embodied in other forms without departing from the spirit or essential 
characteristics thereof. The embodiments disclosed in this application are to be considered in all 
respects as illustrative and not limiting. The scope of the invention is indicated by the appended 
claims rather than by the foregoing description, and all changes which come within the meaning and 
range of equivalency of the claims are intended to be embraced therein. 
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